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This  is  a  working  paper  in  the  true  sense.   Comments  and  criticism 
are  invited.   It  will  become  part  of  a  monograph  of  wider  scope  to  be 
entitled  "Building  Models  for  Decision  Makers:   An  Application  to  Family 
Planning". 


ABSTRACT 

This  paper  draws  on  the  literature  in  management  science, 
behavioral  science,  and  information  systems  along  with  practical  experi- 
ence to  propose  a  process  of  building  models  that  will  be  implemented. 
The  steps  are:  (1)  formulation  of  priors,  (2)  entry,  (3)  problem  finding, 
(4)  specification  of  model  development  criteria,  (5)  model  building, 
(6)  estimation  and  fitting,  (7)  tracking,  and  (8)  continuing  use.  After 
describing  the  state  of  the  art  of  model  implementation,  each  factor 
is  discussed.  The  paper  closes  with  a  preliminary  identification  of 
research  needs  in  this  area. 
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An  Emerging  Process  of  Building  Models 

for  Management  Decision  Makers 
by  Glen  L.  Urban 

Introduction 

The  volume  of  publication  in  Kanageraent  Science  and  Operations  Research, 
numerous  books,  a  multitude  of  seminars,  and  a  substantial  community  of  MS/OR 
professionals  document  the  amount  of  effort  put  into  managerial  model  building. 
But  the  reported  amount  of  implementation  of  models,  managerial  impact,  and 
decision  payoff  is  very  small.   For  example.  Management  Science:   Application 
contained  70  articles  in  1971,  btt  only  3  represented  implementation  in  an 
organization  and  only  10  more  were  applied  even  once  in  a  real  decision 
situation.   In  Operations  Research  the  percentages  are  even  lower.   The 
degree  of  data  basing  is  a  weaker  criteria  of  inclination  toward  application, 
but  even  in  this  dimension  only  40  percent  of  the  articles  in  Management 
Science:   Application  supplied  any  empirical  support  at  all. 

It  is  difficult  to  document  the  exact  level  of  implementation  by 
publication  because  there  is  a  tendency  for  field  practitioners  not  to  report 
really  successful  application?  .since  they  may  be  viewed  as  proprietory, 
or  failures  , since  this  may  imply  incompetence.   As  a  result,  the  literature 
tends  to  include  rather  mediocre  applications  when  it  contains  applications 
at  all.   Private  model  builders  also  place  low  priority  on  publication  and 
along  with  the  delays  and  the  difficult  work  associated  with  publication,  the 
number  of  reported  applications  in  professional  journals  is  minimized.   The 
percentage  of  articles  in  Management  Science:   Application  even  partially 
authored  by  non-academicians  has  been  decreasing  and  was  below  15  percent  m 
1971.   This  trend  is  being  reversed  with  the  advent  of  the  Institute  of 


Management  Science's  Interfaces  vhlch  allows  a  more  convenient  medium 
for  practictioners . 

With  a  lack,  of  full  knowledge  of  implementation ,  it  is  difficult  to 
explicitly  define  the  state  of  the  art,  but  based  on  the  author's  discussion 
with  practitioners  in  companies,  the  literature,  and  management  seminar 
discussions  over  the  last  five  years,  figure  one  seems  to  be  a  reasonable 
description.   Implementation  is  divided  into:    1)   experimental  use  - 
first  trial,   (2)   emerging  use  -  wider  testing,   (3)   continuing  use  by 
one  manager,  and   (4)   institutionalization  -  use  by  many  managers  on  a 
continuing  basis  as  part  of  their  decision  procedure.   Decisions  from  control 
to  strategic  planning  are  represented  on  the  second  dimension.   Tlie  graph 
should  be  interpreted  analogously  to  a  regression  line  based  on  individual 
applications  representing  points.   Very  simple  models 

often  used  to  present  data  to  managers  are  not  included  in  figure  one.   For 
example,  a  pro-forma  balance  sheet  reflects  the  accounting  model  of  the  firm 
and  sales  reports  may  reflect  a  geographic  model  of  the  market.   Many 
firms  use  simple  format  models  to  display  data  to  decision  makers.   Some 
decision  makers  rely  on  such  data  and  use  it  in  their  decision  making.   This 
would  imply  a  high  degree  of  implementation.   It  should  be  noted,  however, 
that  one  is  amazed  at  how  very  little  of  the  data  spued  out  by  "information 
systems"  is  used.   Figure  one  does  not  consider  the  simple  structures  used  to 
retrieve  and  display  data  since  consideration  in  this  paper  will  center 
around  formal  mathematical  models. 

Control  decisions  such  as  determining  inventory  levels  or  distribution 
and  production  control  seem  to  be  the  most  widely  supported  by  models.   Although 
such  applications  are  not  nearly  as  sophisticated  as  they  could  be,  a 
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number  of  firms  are  using  such  control  models  on  a  continuing  basis.  In 
considering  decisions  such  as  production  scheduling,  media  selectiorv  and 
forecasting,  evidence  can  be  found  in  some  firms  of  model  usuage.   This 

is  particularly  true  in  the  oil  industry  where  L.  P.  models  for  production 

1 
planning  are  used.   But,  in  general,  fewer  organizations  use  MS/OR  models 

in  planning  than  control  and  most  are  only  experimenting.   Models  for  the 

most  difficult  decisions  of  strategic  planning  such  as  new  product  development, 

mergers  and  acquisitions,  and  advertising  budgeting  have  achieved  little 

use.   Although  some  firms  have  been  experimenting,  few  firms  use  sophisticated 

models  for  strategic  planning  on  a  continuing  basis.   One  exception  appears 

to  be  Xerox,  which  is  said  to  have  an  integrated  strategic  planning  system 

institutionalized,  but  no  published  reports  seem  to  exist. 

Figure  One  indicates,  as  decisions  from  control  to  planning  are 

considered,  the  degree  model  implementation  drops.   This  correlates  well 

with  the  need  for  more  sophisticated  models  and  the  increasing  absence  of 

problem  structure  in  planning.   The  state  of  the  art  also  varies  by 

functional  area.   The  rank,  order  of  implementation  is  production,  finance, 

and  marketing.   This  rank  order  seems  to  correlate  to  the  percent  of 

structured  versus  unstructured  problems  in  each  area.   In  the  public  sector, 

2 
the  degree  of  implementation  is  even  lower  than  in  the  private  sector. 

Although  McNamara  gained  fame  with  PPBS ,  it  does  not  appear  to  be  an 

institutionalized  procedure  today.   In  softer  areas  such  as  health, 

education,  welfare,  and  transportation  where  organizational  diffuseness, 

lack  of  specific  and  measurable  goals,  poor  data,  and  high  political  content 

characterize  problems,  few  if  any,  models  can  be  cited  as  being  used  on  a 

continuing  basis  by  a  decision  maker. 


In  general  it  seems  that  little  of  the  vast  effort  put  into  MS/OR 
model  building  has  yet  paid  off  in  terms  of  implementation  and  the 
improvement  of  organization  effectiveness.   This  problem  has  not  gone 
unnoticed,  but  it  has  not  received  the  attention  it  deserves.   Less  than 

5  percent  of  the  papers  in  Management  Science  and  Operation  Research 

3 
involve  implementation  research.   Only  one  of  70  papers  in  Management 

Science:   Applications  in  1971  was  directed  at  implementation  problems. 
The  work  in  the  literature  tends  to  examine  special  aspects  such  as  the 

need  for  mutual  understanding  and  communication   the  effects  of  cognitive 
decision  styles  ,  characteristics  of  successful  projects  ,  the  factors 
underlying  implementation  problems  ,  or  the  criteria  for  an  implementable 
model".   Other  works  reflect  narratives  and  "gems  of  wisdom".    There  has 
been  no  integrated  effort  to  define  how  to  build  MS/OR  models  that  will  be 
implemented.   The  largest  body  of  relevant  research  outside  of  mS/OR  is 
in  organizational  development.   Several  procedures  for  planned  change  have 
been  proposed.   The  Klob  and  Frohman  approach  is  the  most  relevant  to  the 
problem  posed  by  this  paper. 

The  purpose  of  this  paper  is  to  draw  on  the  author's  personal  experience, 
informal  discussions  with  other  model  builders  (practitioners  and  academic 
colleagues),  and  the  published  literature  to  advocate  a  process  of  how  to 
build  models  that  will  be  used  by  decision  makers  to  improve  their  organi- 
zations' performance.   This  methodology  is  relevant  to  models  for  decision 
makers  and  does  not  apply  as  well  to  advocate  models  used  to  support 
argumentative  presentations  to  decision  makers.   Such  models  may  be  an 
effective  method  of  influencing  decisions,  but  the  implicit  postulate  of 
this  paper  is  that  models  will  be  most  effective  in  producing  organizational 
payoff  when  they  are  used  as  a  tool  by  the  manager  himself  in  his  decision 
making. 
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Overall  Process 
The  overall  process  for  building  implemen table  models  is  described 
in  figure  two.   It  begins  with  the  consideration  of  priors,  proceeds 
through  entry,  problem  finding,  specif icatimi  of  model  development  criteria, 
model  building,  estimating,  and  tracking  to  end  in  continuing  use.   This 
structure  is  not  meant  to  be  rigid.   In  most  situations  iteration  will  take 
place  with  more  feedback  than  the  irajor  evolutionary  feedbacks  indicate. 
But  the  model  builder  should  be  careful  to  consider  each  step  and  the 
issues  associated  with  it.   It  is  most  important  for  the  model  builder 
who  is  serious  about  implementation  to  consider  application  throughout  the 
process.   Building  a  model  and  then  implementing  it  is  not  a  good  approach. 
The  model  itself  will  be  substantially  different  if  implementation  is 
considered  throughout  the  effort. 

Priors  are  important  to  recognize  since  few  model  builders  realize 
the  biases  their  past  training  and  attitudes  bring  to  a  problem.   Model 
builders  should  have  a  "tool  kit"  o f  OR  and  statistical  approaches  rather 
than  one  pet  technique.   As  a  tool  kit  is  used,  updating  will  occur  and 
posterior  estimates  of  appropriateness  will  be  formed.   Good  prior  thought 
about  the  specific  problem  area  is  necessary  for  the  model  builder  to 
legitimize  himself  to  managers  and  it  sometimes  requires  "selling"  to 
gain  entry.   Care  should  be  taken  in  determining  the  entry  point  so  that 
decision  leverage  can  be  gained.   At  entry,  education  efforts  should  be 
carried  out  since  this  will  facilitate  communication  and  lead  to  effective 
problem  finding.   Most  model  builders  attack  the  wrong  problems  (at  least 
from  the  managers  point  of  view)  so  substantial  effort  should  be  allocated 
to  identify  the  correct  first  project.   Tnis  calls  for  empathy  on  the 
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FIGURE  TWO: 
Emerging. Process  of  Building  Models  for  Decision  Makers 
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model  builder's  part  and  a  diffuseness  of  his  priors  that  will  allow  him 
to  accept  problem  definitions  different  from  his  prior  thought. 

Given  an  appropriate  problem,  specific  model  building  criteria 
pertaining  to  implementation,  model  specification  (factors,  phenomena, 
and  variables),  and  data  should  be  formulated.   These  become,  along  with 
resource  constraints,  the  basis  of  selecting  the  level  of  detail  for  the 
model  and  the  model  type  most  appropriate.   It  is  important  to  consider 
alternate  model  forms  at  this  point  and  make  explicit  modeling  and 
managerial  tradeoffs.   With  a  first  model  structure,  data  is  examined 
by  linear  and  non-linear  statistics.   Empirical  data  is  combined  with 
judgment  to  fit  past  data.   Lack  of  fit  may  require  feedback  to  re-define 
the  model  criteria  or  structure.   Since  managers  sometimes  do  not  know 
what  they  want  until  they  see  it,  tne  re-cycling  may  go  on  as  far  back  as 
problem  revision. 

After  an  appropriate  managerial  model  has  been  shown  to  fit  past  data 
well,  it  can  be  further  tested  by  use  in  the  decision  process.   Such  use  will 
generate  forecasts  which  over  time  can  be  compared  to  actual  results.   This 
process  is  called  "tracking"  and  includes  a  diagnosis  to  find  if  the  para- 
meters or  model  structure  need  to  be  updated.   The  diagnosis  may  also 
identify  new  problems  that  will  require  other  models  or  elaboration  of 
the  existing  model. 

The  final  step  in  the  modeling  process  is  continuing  use.   This  is  a 
process  of  widening  use  within   the  organization  and  institutionalizing  it  as 
decision  procedure.   Continued  tracking  will  probably  lead  to  new  model 
evolution,  other  models,  and  an  increased  confidence  in  the  validity  of 
the  model  structure. 


Each  of  the  elements  will  now  be  discussed  in  detail  and  this  paper 
will  end  with  a  brief  indication  of  tests  of  this  methodology  and  future 
research  needs. 

Formation  of  Priors 
Few  model  builders  realize  the  strength  of  their  priors  even  though 
they  can  clearly  see  others  who  are  biased  towards  one  particular  approach. 
These  favored  approaches  may  be  specific  techniques  like  math  programming 
and  econometrics  or  functional  orientations  of  marketing,  production,  or 
finance.   Some  researchers  seem  to  find  all  problems  fit  into  a  particular 
model  structure  like  a  Monte  Carlo  or  industrial  dynamics  simulation. 
Figure  3  represents  the  conviction  that:   (1)   problems  should  lead  to 
models,   (2)   solutions  are  gained  by  applying  techniques  (e.g.,  simu- 
lation, math  programming,  heuristics)  to  the  models,  and   (3)   data 
interpreted  through  statistical  models  are  used  to  estimate  models.   In 
this  view  the  researcher  should  have  a  tool  kit  of  OR/MS  and  statistical 
techniques,  as  well  as,  a  number  of  alternate  model  building  approaches 
so  that  he  can  effectively  solve  problems.   This  is  in  contrast  to  the 
approach  of  taking  a  technique  like  math  programming  and  forcing  the 
problem  to  fit  it  or  taking  a  statistical  approach  like  regression  and  ab- 
stracting the  problem  aspects  that  can  be  analyzed  by  this  procedure.   It 
is  best  to  be  capable  of  applying  all  significant  techniques  and  to 
utilize  the  appropriate  one  in  a  given  problem  setting. 

It  is  not  undesirable  to  have  priors  on  technique  and  approach,  but 
care  should  be  taken  to  initially  build  rather  diffuse  priors  and  a  wide 
range  of  skills.   Initial  training  for  model  builders  should 
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include  the  basic  disciplines  of  economics,  behavioral  science  (especially 
organizational  and  consumer  behavior),  quantitative  methods  (OR  and  statistics), 
and  functional  management  approaches.   This  should  be  followed  by  actual 
model  building  experiences  in  real  environments,  as  well  as,  the  study 
of  a  range  of  existing  models.   This  should  provide  the  student  with  a 
set  of  capabilities  which  can  then  be  modified  by  experience.   Evidence 
indicates  that  in  general  people  are  too  conservative  in  their  updating  of 
priors.     Model  builders  should  be  sensitive  to  experience  and,  through 
applications  in  many  problem  situations,  build  an  effective  tool  kit  of 
alternate  modeling,  statistics,  and  solution  capabilities.  If  priors  wcv^. 

correctly  formed  by  initial  training,  experience  should 

lead  the  model  builder  to  update  his  priors  so  that  Le  has  a  portfolio 

of  approaches  that  could  work  well  and  a  feeling  for  the  kind  of  problem 

areas  where  he  can  have  impact. 

It  is  usually  necessary  to  have  fairly  concrete  priors  if  the 
model  builder  wants  to  enter  a  real  environment,  since  he  will 

have  to  sell  himself  and  establish  his  credibility.    Tliese  specific  priors 
can  be  generated  by  studying     the  decision  environment,  talking  to 
people  in  the  field,  and  fron  the  literature.   The  model  builder  should 
consider  a  range  of  general  models  that  could  contribute  to  the  real  problem 
area.   With  this  thinking  and  his  past  achievements,  he  should  be  able 
to  make  an  effective  entry. 


Entry 

The  process  of  entry  into  the  organization  is  very  important.   It  is 
recommended  that  entry  be  made  at  the  decision  point.   This  is  where  the 
leverage  is  and  if  the  model  is  to  be  used  by  the  manager,  it  is  best  to 
work  with  the  decision  level  as  soon  as  possible.   A  good  first  contact 
and  an  essential  activity  is  an  educational  program  about  models,  manageme-nt 
science,  and  information  systems.   This  should  be  top  management  oriented 
and  could  be  done  in  a  one  day  seminar  at  the  office  or  in  conference  at 
a  "pleasant"  location  away.   This  education  should  include  discussion  of 
terms  and  other  applications,  but  emphasize  specific  "hands  on"  model 
experience.   In  some  companies  the  educational  task  already  has  been 
carried  out,  so  informal  tutorial  sessions  with  each  manager  may  be  most 
appropriate.   In  others  where  there  is  no  experience,  some  very  brief 
project  to  build  a  simple  model  may  be  needed  to  provide  relevance  for 
the  educational  effort  and  to  demonstrate  the  capability  to  understand 
the  individual  organization's  problems. 

If  the  educational  effort  is  unsuccessful  and  management  is  willing 
to  look  into  the  potential  of  models,  a  team  should  be  formed  with  the 
relevant  decision  maker  as  its  leader.   The  team  should  be  small,  but 
include  in  addition  to  a  manager,  one  or  two  staff  people,  and  a  change 
agent.   The  change  agent  is  particularly  important.     He  is  a  man  who 
can  effectively  work  in  the  informal  organization  to  advocate  and  gain 
acceptance  of  the  innovation  of  models.   His  capabilities  must  be  greater 
in  organizational  development  than  in  model  building.   Tliis  team  must  be 
backed  by  top  management  support.   Tlie  president  must  approve  of  the  work 
and  decision  makers  must  perceive  that  the  effort  they  put  into  the  project 
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will  be  valued. 

In  addition  to  top  management  support,  a  commitment  of  time  and  funds 
must  be  made.   The  time  commitment  by  managers  inside  and  outside  the  team 
to  aid  in  model  building  is  the  most  important,  although  initial  funding 
is  not  trivial  in  many  cases.   Particularly  in  public  sector  work,  "seed 
money"  is  an  effective  way  to  fund  feasibility  and  initial  work.   But  even 
in  these  cases  it  is  advisable  to  have  the  organization  budget  some  funds, 
however  small,  since  this  builds  commitment,  interest,  and  evaluation  into 
the  project.   The  model  builder  must  be  careful  not  to  orient  his  work 
towards  the  funder,  but  rather  towards  the  manager  who  will  ultimately  make 
the  project  pay  off  by  improved  decision  making. 

Problem  Finding 

Given  the  commitment  and  resources  to  pursue  model  building,  effort 
should  be  concentrated  on  finding  the  most  appropriate  problem.   Priors 
may  have  been  used  to  identify  potential  problems  and  to  form  the  team 
membership,  but  now  a  substantial  effort  should  be  made  to  find  the  best 
problem  to  work  on  first.   Experience  indicates  14  to  21  man  days  is 
sometimes  necessary  to  effectively  define  the  correct  problem.   Many  modelers 
skip  over  this  problem  finding  step  and  assume  they  have  a  good  problem 
definition  only  to  find  at  completion  of  the  project  that  it  was  not  the 
problem  at  all  or  at  least  not  the  one  that  interested  the  manager. 

A  good  way  to  find  problems  and  understand  the  organization's  needs  is 

to  carry  out  a  descriptive  study  to  determine:   (1)   existing  models  or  rules 

13 
of  thumb,     (2)   the  characteristics  of  the  decision  process  (who  makes 

decisions,  when,  and  on  what  basis?),   (3)   the  existing  flow  and  usuage  of 

information,   (4)   the  clientele 's  behavior  and  relationship  to  the  organization, 

(5)   the  stated  and  apparent  goals  of  the  organization,  (6)   the  informal  and 

formal  organization  structure  (including  power,  friendship,  and  authority 
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relationships)  and   (7)   managers'  definitions  of  their  perceived  problems. 
This  could  be  done  by  extensive  interviews  with  managers,  a  study  of  recent 
and  on-going  decisions,  probing  questionnaires,  and  consideration  of  managers' 
self  designated  perceptions.   This  process  calls  for  a  great  deal  of  empathy 
on  the  model  builder's  part  and  a  willingness  to  revise  his  prior  designation 
of  problems.   He  must  be  sensitive  to  their  needs  and  stated  problems,  but 
he  must  realize  that  managers  may  not  know  what  their  problems  really  are. 
For  example,  a  manager  who  is  experiencing  a  profit  decay  may  feel  it  is  e. 
cost  problem  when,  in  fact,  the  marketing  strategy  is  the  basis  of  the 
problem.   If  the  model  builder  perceives  a  different  problem  it  would  be 
wise  to  start  with  the  manager's  problem,  but  have  an  explicit  evolutionary 
plan  that  will  lead  to  the  manager's  gaining  a  perception  of  the  wider  problem. 

Rockart  has  found  that  comparison  of  a  normative  model  generated  by  the 
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analyst  when  compared  to  existing  procedures  can  help  find  problems. 

In  a  clinic  scheduling  problem,  he  found  a  coraparision  of  a  normative  job 

scheduling  model  to  existing  procedures  Identified  over  two-thirds  of  the 

problems  eventually  found  by  an  extensive  problem  definition  exercise. 

The  problem  finding  study  will  yield  many  topics  and  the  selection  of 

the  first  project  must  be  made.   In  general,  it  should  be  a  "simple"  problem 

that  can  be  done  within  the  existing  state  of  the  art  of  MS/OR  and  one  that 

can  be  easily  understood  by  managers.   It  should  be  a  significant  and  visible 

problem.   Since  managers  will  have  to  devote  time  to  this  project  it  must 

justify  such  effort.   Visibility  is  important  in  gaining  recognition  and  a 

base  for  future  work.   A  good  rule  is  to  work  in  an  area  where  a  large  budget 

is  involved  so  that  gains  from  the  model  building  will  be  large.   The  first 

project  should  be  low  cost  and  produce  some  results  in  a  reasonable  time  (less 
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than  six  months).   It  should  involve  managers  and  be  interesting  to  them. 
Finally,  it  is  desirable   to  work  in  an  area  where  relevant  data  exists. 
Clearly,  it  is  only  in  a  few  cases  where  all  these  criteria  can  be  met. 
The  highest  priority  is  on  managerial  interest  and  involvement,  assuming 
the  model  builder  can  do  the  job. 

After  selecting  the  project,  a  specific  psychological  contact  should 
be  made  between  the  managers  and  team  members  that  includes  an  under- 
standing of  goals  of  the  project,  individual  goals,  resource  commitments, 

the  relationship  between  members  and  the  organization,  and  the  specific 
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scope  of  the  work.     It  is  useful  at  this  point  to  visualize  the  likely 

outcome  and  carry  out  a  rough  cost  benefit  analysis.   An  exercise  of 
this  is  to  try  to  write  out  the  memo  that  would  be  sent  to  top  management 
after  one  year  of  the  project  to  document  real  gains  and  payoff.   At  thic- 
point  the  team  composition  may  also  be  revised.   Since  it  is  important  to 
have  the  real  decision  maker  on  the  team,  the  manager  involved  in  the 
project's  decision  may  be  added  as  the  new  team  captain  or  as  a  member.   In 
some  cases,  a  whole  new  team  may  be  formed  since  the  first  team  had  per- 
formed its  functions  of  organizational  legitimization  and  problem  finding. 

Model  Development  Criteria 
Given  a  good  problem  and  a  clear  understanding  of  the  project  activities, 
specific  model  development  criteria  should  be  formulated.   First,  is  the  model 
to  be  descriptive,  predictive,  or  normative?   A  model  that  forecasts  well 
may  be  very  different  in  structure  from  a  normative  model.   An  exponential 
smoothing  model  may  be  a  good  forecasting  model,  but  be  useless  in  deter- 
mining what  level  of  advertising  and  salesman  compensation  should  be  set. 
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Next  the  factors,  phenomena,  and  variables  important  in  the  problem 
should  be  listed  and  rank  ordered  in  terms  of  priority  for  inclusion  in 
the  model.   Particularly,  the  phenomena  of  consumer  or  clientele  behavior 
to  be  included  should  be  explicitly  considered.   The  physical,  organizational^ 
and  external  constraints  should  be  listed  and  the  opportunity  for  data 
basing  examined.   Model  development  criteria  should  be  set  in  terms  of 
inclusion  of  variables,  factors,  and  phenomena  along  with  policies  stated  in 
terms  of  constraints  and  a  view  towards  the  degree  of  empirical  content. 
In  the  case  of  normative  models,  the  criterion  to  be  used  to  define  "best" 
or  "optimal"  should  be  defined,  if  possible.   In  some  cases  meaningful 
goals  cannot  be  specified  until  the  model  is  structured  and  managers  have 
interacted  with  it  and  increased  their  level  of  perception.   Finally,  the 
criteria  should  include  specification  of  the  evolutionary  steps  that  will 
lead  to  the  described  final  model.   ITiese  criteria  will  be  important  in  the 
tradeoff  between  alternative  models. 

In  addition  to  criteria  relating  to  model  purpose  and  scope,  explicit 
implementation  criteria  should  be  set  for  the  model.   Little  has  oroposed  a 
set  of  criteria  for  models  that  will  be  used  by  managers.   He  suggests  a 
model  be:   (1)   understandable,   (2)   complete,   (3)   evolutionary,   (4) 
easy  to  control,   (5)   easy  to  communicate  with,   (6)   robust,  and  (7) 
adaptive.    These  criteria  will  be  reviewed  here  with  a  few  changes  from 
Little's  work. 

The  first  criterion  -  understandable  -  requires  a  simple  model  structure 
that  is  perceptibly  similar  to  how  the  manager  thinks.   For  example,  process 
flow  models  are  usually  intuitively  acceptable  to  managers,  while  generally  a  lineai 
programming  equation  is  not.   If  the  model  is  not  a  process  model,  it  will 
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be  more  understandable  if  it  can  be  represented  graphically  and  with  a 
minimum  of  mathematics.   Understanding  applies  not  only  to  the  manager 
formulating  a  plan,  but  to  the  levels  in  the  organizational  hierarchy 
which  review  plans.   If  the  model  is  not  understandable  to  middle  and  top 
management  they  probably  will  not  accept  the  new  decision  procedure,  and 
the  model  will  not  be  an  effective  communication  mechanism. 

The  second  criterion  is  completeness  and  it  demands  that  none  of 
the  factors  or  phenomena  important  to  the  manager's  definition  of  the 
problem  are  omitted.   Tliis  is  in  conflict  with  the  first  criterion  since 
most  decisions  are  complex  and  completeness  implies  a  model  that  is  difficult 
to  understand.   Criterion  three  allows  an  escape  from  this  conflict.   If 
a  simple  model  can  evolve  to  a  more  complex  model  in  an  orderly  manner,  the 
manager  can  be  taught  the  basic  structure  at  a  simple  level  and  then  evolve 
to  a  complete  and  more  analytical  model.     The  evolutionary  criterion  also 
implies  building  a  model  that  can  be  generalized  to  many  situations  from 
the  initial  application  environment. 

Perhaps  the  most  important  criterion  is  control.   Tlie  manager  must 
feel  it  is  "his"  model  and  that  it  reflects  a  refined  statement  of  his 
implicit  model.   The  model,  if  it  is  to  be  used  on  a  continuing  basis,  must 
be  viewed  as  a  necessary  tool  for  his  work.   He  must  not  feel  that  in  using 
the  model  he  is  delegating  his  decision  prerogatives,  but  rather  that  the 
model  is  helping  him  to  exercise  them.   The  manager  should  control  the  model 
to  the  extent  that  he  could  manipulate  the  model,  if  he  so  desires.   That 
is,  he  should  know  the  inputs  necessary  to  get  specific  outputs. 

A  complete  and  understandable  model  encourages  the  feeling  of  control, 
but  the  use  of  on-line  computers  has  made  it  feasible  for  the  manager  to 
directly  interact  with  a  model.   Tlie  use  of  a  good  conversational  program 
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and  graphical  display  allow  a  manager  to  communicate  easily  with  the  model 
from  his  office  via  a  remote  terminal.   Tliis  element  of  control  and  ease 
of  use  are  important  to  the  internalization  process. 

The  process  of  acceptance  can  be  reinforced  by  a  robust  model.   The 
model  should  not  produce  wildly  unexpected  results.   For  example,  a  linear 
advertising  budgeting  model  is  not  robust  since  it  will  lead  to  an  optimal 
level  of  advertising  at  plus  infinity  or  zero.   The  model  should  be 
structured  to  give  reasonable  answers  and  the  output  be  restricted  to 
reasonable  ranges  such  as  the  limits  of  the  data  used  for  estimation.   In 
using  a  robust  model,  the  manager  gains  confidence  and  is  not  distracted 
by  inappropriate  asymptotic  outcomes. 

Given  a  model  that  meets  the  above  criteria,  the  next  question  is  how 
the  model  parameters  will  be  obtained.   A  model  will  be  more  heavily  used 
if  its  parameters  are  data  based.   Initially,  in  order  to  meet  the  com- 
pleteness criterion,  it  will  obviously  be  necessary  to  rely  heavily  on 
subjective  inputs,  but  the  use  of  the  model  will  aid  the  manager  in  specifying 
his  data  needs  and  encourage  him  to  use  the  data  after  it  has  been  collectad. 
In  some  cases,  historical  regression  is  informative  about  parameters,  but 
in  most  models  where  causual  inferences  are  required,  they  can  be  gained 
best  by  experimentation.   The  most  ideal  data  basing  system  is  one  that 
utilizes  continuing  experimentation  and  permits  adaptive  control.   In  this 
way,  the  model  can  be  updated  to  reflect  changes  in  response  while  the 
continual  updating  and  refining  will  build  managerial  confidence  in  the  new 
decision  structure. 

During  the  parameterization  and  updating  of  the  model,  it  is  important 
to  allow  managerial  judgment  to  enter  the  process.   The  model  should  be 
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structured  so  that  the  manager  can  relate  his  experience  to  the  parameters. 
It  may  seem  unnecessary  to  require  this  if  experiments  are  conducted,  but 
experiments  tend  to  be  confounded  by  uncontrollable  factors  (e.g.,  competitors 

and  local  market  changes)  and  the  causative  effects  are  sometimes  small 

18 
relative  to  the  variance  in  sales.    Tliis  means  the  statistical  results 

may  require  interpretation  before  use  in  a  normative  model.   This  inter- 
pretation is  facilitated  if  the  manager  understands  the  model. 

After  specifying  the  decision  calculus  criteria  for  an  implementable 
model,  a  set  of  criteria  on  completeness  and  inclusion  of  factors,  phenomena, 
and  variables,  a  definition  of  the  constraints,  and  a  policy  on  data  basing, 
model  construction  can  begin. 

Model  Building 

Model  building  tends  to  possess  the  characteristics  of  an  art.   The 
model  builder  has  available  many  alternate  forms  and  he  tries  to  interpret 
and  structure  a  somewhat  nebulous  problem  with  them.   In  the  same  situation 
some  people  can  create  beautiful  models  which  represent  perceptive  structuring 
of  previously  unstructured  problems,  while  others  shape  awkward  and  uncon- 
vincing models. 

Some  emerging  ideas  to  help  at  least  position  alternative  model  forms 
will  now  be  discussed.   With  this  classification  of  model  types  and  the 
previously  specified  model  development  criteria,  the  most  powerful  structures 
should  be  evident.   Within  this  restricted  set,  structuring  and  alternative 
model  tradeoffs  should  be  easier. 

Table  one  shows  decision  models  divided  into  aggregate  response, 
hierarchical,  and  perceptive  groups.   The  aggregate  response  models  are 
models  based   upon   an  independent  variable  being  a  function  of  a  set  of 
dependent  variables.   These  tend  to  be  multiple  and  sometimes  simultaneous 
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equation  sets.   They  model  the  response  of  each  dependent  variable  on  the 
independent  variable  and  changes  in  one  variable  can  compensate  for  changes 
in  another.   Math  programming  and  econometrics  are  most  commonly  used  to 
analyze  these  models.   The  second  major  class  of  models  are  hierarchical, 
rhey  are  either  sequential  decision  or  flow  type  models . Events  do  not  occur  at  one  time  as 
in  aggregate  response  models,  but  rather  occur  in  an  ordered  sequence. 
Examples  are  Monte  Carlo  simulation,  industrial  dynamics,  or  other  macro 
flow  models  which  are  structured  as  dynamic  and  deterministic  flows.   Tiiese 
first  two  classes  are  further  divided  into  stochastic  and  deterministic 
sections.   The  third  type  of  model  is  called"perceptive"  because  their  major 
purpose  is  to  reveal  new  insights  for  management.   Examples  are  market  structure 
models  based  on  non-metric  multidimensional  scaling,  information  processing 
models,  or  organizational  models.   It  should  be  noted  that  aggregate  response 
and  hierarchical  models  also  help  managers  gain  new  insights  by  explicitly 
modeling  the  decision  environment.   The  footnotes  to  Table  one  cite  some 
specific  examples  of  each  type  of  model.  The   table  does  not  name  all  existing 
types  of  models,  but  it  is  hoped  that  it  contains  the  major  model  alternatives. 

Considering  the  development  criteria,  the  choice  of  perceptive 
models  is  possible  only  if  the  criterion  was  to  build  a  descriptive 
model,  so  the  most  difficult  choice  is  between  aggregate  response  and 
hierarchical  models.   The  basic  nature  of  a  problem  may  clearly  indicate  one 
approach.   For  example,  since  people  flow  through  a  medical  system,  a 

sequential  model  is  attractive.   But  aggregate  models  have  also  been  used 

33 
to  analyze  health  systems.     While  the  problem  itself  will  help  in  the 
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TABLE  ONE: 

A  Classification  of  Decision  Model    Structures 


Aggregate  Response 

19 
Deterministic 

Linear  -  non-linear,  univariate  -  multi-variate,   static 

dynamic,  and/or  single  -  simultaneous  equations 

Stochastic 

20 
Risk  analysis 

Econometric 

22 
Expected  response 

Hierarchical 

Deterministic 

23 
)w  models 

24 


23 
Macro  flow  models 


Industrial  (systems)  dynamics 
Sequential  analysis  -^ 


Stochastic 

Monte  Carlo  models 

27 
Markovian  Models 

28 
Queue ing 

Bayesian  decision  analysis 


Perceptive 


30 
Market  structure  models 

31 
Attitude  scaling 

32 
Information  processing 
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choice,  the  basic  level  of  detail  needed  is  an  important  determinant.   In 
general,  the  flow  type  models  are  ir.ore  detailed  than  aggregate  response 
models,  but  within  each  class  different  levels  of  detail  are  possible. 
Consider  the  level  of  detail  tradeoffs  shown  in  figure  4.  As  the  level 
of  detail  increases,  the  model  is  more  difficult  to  estimate,  solve,  and 
test,  but  a  richness  expressed  as  behavioral  content  in  consumer  models  or 
feedback  dynamics  in  system  models  is  gained.   Tlie  specification  of  the 
model  development  criteria  relating  to  the  phenomena  and  factors  to  be 
included  in  the  model  helps  in  the  choice.   If  a  high  degree  of  heterogeneity 
and  feedback  response  must  be  encomoassed,   a   flow  model  will  be  appropriate. 
If  more  macro  and  compensatory  phenomena  are  to  be  modeled,  then  the 
aggregate  response  model  is  appropriate. 

The  most  important  factors  in  the  choice  of  the  model  are  reflected  in 
the  decision  calculus  criteria.   Does  the  manager  basically  see  it  as  a  flow 
or  a  response?  Which  type  is  most  compatible  with  his  existing  or  implicit 
model?   It  is  strongly  recommisnded  that  the  model  be  very  compatible  to  the 
manager's  view  of  the  decision  environment.   The  model  should  evolve  from  a 
simple  version  roughly  similar  to  his  implicit  model  to  a  more  complete 
and  powerful  version.   This  initial  version  must  feel  comfortable  to  the  manager 
if  he  is  to  understand  it  and  he  must  feel  that  he  controls  it.   It  must  be 
remembered  that  the  new  model  will  be  in  competition  with  his  old  model. 

The  choice  between  deterministic  and  stochastic  models  may  be  dictated 
by  the  development  criteria  of  the  problem,  but  in  general,  managers  have 
enough  trouble  understanding  deterministic  concepts.   It  is  suggested  that,  in 
most  cases,  stochastic  aspects  be  added  well  down  the  path  of  evolutionary 
development. 
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FIGURE  FOUR 
Some  Level  of  Detail  Tradeoffs 
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Final  considerations  in  the  choice  of  level  of  detail  are  the  budget, 
time,  and  personnel  available.   In  general,  more  detailed  models  cost 
more,  require  more  managerial  time,  need  more  real  time  to  complete,  and 
are  more  subject  to  problems  of  personnel  turnover.   To  the  extent  that 
flow  models  initially  or  during  evolution  are  more  detailed,  the  aggregate 
response  models  would  be  indicated  if  budget  and  time  constraints  are 
severe. 

It  is  clear  from  the  previous  paragraphs  that  a  great  deal  of  art 
remains  in  model  building.   It  is  hoped  that  the  classification  of  model 
types  and  level  of  detail  dimension  interpreted  through  the  model  development 
criteria  will  help  restrict  the  search  to  a  few  relevant  alternatives. 
Within  this  set,  models  should  be  roughed  out  for  each  alternative  type 
and  evaluated  by  the  model  builder  and  manager  to  make  a  decision  on  the 
best  approach.   Given  the  best  model  structure,  the  researcher  must  refine 
the  interpretation  of  the  problem  in  the  model  and  satisfy  the  model 
development  criteria.   This  process  can  be  facilitated  by  continued 
interaction  between  the  model  builder,  manager,  and  the  development  team. 
During  this  process,  the  model  usually  grows  too  large  and  complex  and 
continuing  pressure  must  be  applied  to  keep  the  model  understandable  for 
managers.   This  can  be  done  by  setting  priorities  on  complexity  and  developing 
an  explicit  evolutionary  development  plan.   As  the  first  simple  but 
meaningful  model  is  developed,  it  can  be  combined  with  data  and  judgments 
to  lead  to  initial  use. 

Estimation  and  Fitting 
Data  for  model  building  can  come  from  subjective  judgment,  the 
analysis  of  past  data,  or  experimentation.   The  statistical  analysis  of 
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past  data  is  best  understood.   Linear  regression  procedures  can  find 
the  best  fitting  parameters  given  the  assumptions  of  the  statistical  model. 
Many  times,  however,  the  simple  linear  model  is  not  an  appropriate 
representation  of  the  decision  model  so  that  transformations,  lagged 
terms,  or  simultaneous  equations  must  be  used.   This  raises  a  number  of 
statistical  issues  of  bias  and  consistency,  so  model  builders  must  be 
careful  not  to  inappropriately  apply  regression.   Carrying  out  a  synthetic 
data  analysis  can  help  determine  Che  bias  that  might  result  from  applying 
simple  procedures  to  complex  models.   For  exair.ple,  artificially  generating 
data  from  a  hypothesized  nonlinear  lagged  model  and  repeated  regressions 

on  generated  data  can  be  used  to  escimate  the  bias  in  the  estimates  for  a 
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given  sample  size.     If  the  bias  is  acceptable  in  this  synthetic-setting, 

ordinary  least  squares  regression  could  be  used.   In  simultaneous  equations 

the  situation  is  more  complex  since  procedures  demand  that  "identification" 

conditions  be  met.   But  in  many  managerial  models  (e.g.,  industrial  dynamics) 

the  number  of  exogenous  variables  is  small  and  systems  are  under-identified. 

Some  modelers  add  exogenous  variables  to  provide  indentifiability ,  but  this 
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is  a  dangerous  method  of  achieving  the  statistical  corapatability.    It  is 

not  good  practice  to  add  variables  that  were  not  hypothesized  in  the  decision 

model  unless  a  new  justification  for  inclusion  is  found.   The  statistical 

procedures  should  not  over  rule  the  decision  model  and  its  managerial  relevance, 

In  regression  analysis  various  transforms  are  available  and  in  some 

situations  can  be  used  to  create  decreasing  returns,  hyperbolic,  or  even  "S" 

shaped  responses.   These  transformed  models  are  called  intrinsically  linear. 

Not  all  models  can  be  transformed  ,50  non-linear  estimation  procedures  may  be 

necessary  to  minimize  the  squared  error  or  maximize  a  likelihood  function. 

These  techniques  are  indirect  procedures  based  on  solving  the  set  of  partial 
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differential  equations  or  direct  computer  search  routines.     Although 
these  procedures  impose  few  conditions  on  the  model,  with  the  exception  of 
maximum  likelihood,  they  supply  no  statistical  output  and  the  analyst  must 
interpret  the  significance  of  the  explained  variation  and  the  coefficients. 

Most  historical  data  in  managerial  settings  is  not  very  clean.   Typically 
the  variations  in  variables  is  small  and  data  tends  to  be  heterogeneous.   For 
example,  in  marketing  the  data  base  may  be  time  series  of  sales  data  and 
cross  sectional  attitude  surveys.   The  best  method  of  collecting  data  is 
by  experimentation,  but  this  is  expensive,  and  sometimes  it  does 
not  yield  the  significant  results  desired.   In  most  situations  the  data 
analysis  reveals  gaps,  inconsistencies,  and  a  fairly  wide  variance  on  para- 
meter estimates,  so  subjective  estimates  must  be  used.   It  is  better  to  use 
subjective  judgment  than  to  sacrifice  the  completeness  of  the  model  necessary 
for  managerial  implementation.   If  an  important  parameter  is  not  known,  the 
manager  will  request  relevant  data  since  he  can  see  through  the  model  how 
it  would  help  him. 

Subjective  estimates  may  be  combined  with  the  empirical  estimates  by 
"fitting"  the  models  dependent  variables  to  a  past  set  of  values.   This  is 
facilitated  by  an  on-line   model  since  the  manager  then  can  make  his 
estimates  and  see  their  effect  in  terms  of  improving  the  fit.   This  type  of 
procedure  is  manipulative  and  enough  degrees  of  freedom  usually  exist  to 
perfectly  fit  the  data.   It  is  important  for  the  manager  to  consider  only 
reasonable  parameter  changes  and  justifications  should  be  sought  for  various 
estimates.   The  model  builder  must  remember  that  the  judgments  made  in  fitting 
will  be  tested  by  prediction  in  the  cracking  stage.   It  is  good  practice  to 
save  some  past  data  for  predictive  testing,  if  enough  data  exists.   Then  a 
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test  of  the  quality  of  fitting  will  exist  before  usage. 

Based  on  the  fitting  exercise,  the  model  may  require  revision  due 
to  a  lack  of  fit  or  the  discovery  of  new  phenomena  or  factors.   Man- 
agerial interaction  at  this  level  may  also  generate  new  model  needs  or 
even  problem  elaborations.   These  adaptive  requirements  demand  that  the 
model  be  implemented  in  a  very  flexible  computer  program  so  it  can  be 
readily  revised  by  adding  new  parameters  and  functions  with  a  minimum  of 
time  and  cost. 

Given  the  evidence  that  the  model  adequately  fits  past  data,  it  can 
be  used.   This  initial  usage  is  usually  on  a  small  scale,  but  should  involve 
real  decision  level  interaction.   This  use  will  ]ead  to  decisions  and  fore- 
casts and  provide  a  basis  for  tracking  and  expanded  usage. 

Tracking 
In  tracking,  the  forecasts  of  future  events  are  compared  to  actual 
results  and  the  differences  must  be  reconciled.   They  may  be  due  to:  (1) 
errors  in  forecasting  model  inputs,   (2)   inappropriate  parameter  estimate^  (3)   incor- 
rect model  structuring   (4)   changes  in  the  real  environment  being  modeled, 
or   (5)   random  variation.   Separating  these  causes  for  differences  between 
predicted  and  actual  requires  a  great  deal  of  judgment.   First,  one  can 
see  if  input  forecasts  are  in  error  by  comparing  predicted  and  actual  input 
levels.   Re-running  the  model  with  the  actual  input  will  remove  this 
first  source  of  error.   Parameters  should  be  re-estimated  for  the  new 
period  and  formal  Bayesian  or  informal  procedures  used  to  update  the  para- 
meters.  Re-running  with  the  updated  parameters  and  subjective  fitting  will 
assess  the  adequacy  of  current  model  structure.   In  many  situations  it  is 
found  that  either  the  initial  structure  was  not  complete  (e.g.,  no  competitive 


25. 

price  variable)  or  che  world  changed  (e.g.,  price  controls).   The  recon- 
ciliation is  a  re-fitting  of  the  model  to  the  new  data  and  is  an  updating 
of  assumptions  that  will  allow  a  new  conditional  prediction.   This  adaptive 
use  of  the  model  should  be  continuous  and  will  lead  to  confidence  by  managers 
in  the  model  and  establish  the  model's  "validity". 

The  discipline  of  reconciling  predicted  and  actual  results  often 
identifies  new  problems.   For  example,  the  difference  between  actual  and 
predicted  sales  may  be  due  to  out  of  stock  condition  in  one  distribution 
sub-channel  or  a  change  in  competitive  strategy.   The  manager's  interaction 
in  the  tracking  gives  him  an  effective  problem  finding  and  control  mechaniLsm 
and  often  leads  to  the  specification  of  new  models  or  data  collection  needs. 

Continuing  Use 
As  a  model  is  used  over  time,  tracking  increases  managerial  confidence 
in  the  model  and  usually  leads  to  elaboration  and  evolution.   The  evolutionary 
progress  is  almost  continuous  with  not  only  model  changes,  but  new  model 
needs  being  identified.   This  leads  to  a  model  bank  which  is  a  set  of 
compatable  models  designed  to  meet  special  decision  and  managerial  needs. 
The  data  necessary  to  support  these  models  should  evolve  as  the  models  are 

developed.   Tlie  growth  of  the  data  bank,  model  bank,  and  statistical  cap- 

37 
abilities  should  be  balanced.     The  view  of  this  author  and  others  is 

38 
towards   model  based  information  systems.     Figure  five  describes  an 

example  of  a  model  based  time  pattern  of  growth  leading  to  a  total  decision 

information  system.   This  is  in  contrast  to  efforts  to  build  initially  a 

"total"  system  which  generally  have  not  been  successful.   It  is  also  contrary 

to  the  notion  of  building  the  data  base  first  and  represents  the  view 

stated  by  John  Little  that"the  model  is  the  stone  in  the  shoe  of  data." 


>» 
in 


«/) 

c 
o 


>1 


fO 


o 


o 

o 


>> 


q; 


ex 
=)      o 

U-        > 
Q 


o 

> 


c: 


C3 


I-  Q 

CO  Q  Z 

>-  ^  oo  <C 

1/1  Occ<c<->  ozr_i  ^       LJ 

I —  I —  —  LUOUJ  O   lOO 

_i  LUQoooo  2:1—0  I—  aca 

<:  >       _j— •  >— ;3s:  00 

(_  _lLiJuJt—  I—   —I  — iSIct 

O  OCtQct  ZOC3  o         t— 

K-  r>OOI—  0>z  >u-ci; 

=  uJ2;2:oO  cjLucvj  UJOQ 


o 


I—  —J  <c 

00  Q3  U) 

a;  o  :3 

^  c::  a 

u.  a.  uj 


o 

0) 

Q. 
E 
la 


26. 

During   continuing   use,   efforts   are  made    to   diffuse   the   innovation 
of  models    to    other  managers.       It    is    particularilv    important    to   gain   acceptance 
at   the  middle   level  where   the   experience  of   many   past   years   has   left   a 
sometimes   inflexible   implicit  model   in  managers  minds.      It   is  easier   to 
gain   acceptance   at   the   operating  manager   level   since    these   people   are 
generally   younger   and  more    innovative.       It    is    also   easier   at    the    top    level 
where  organization   innovation   is   at   least   stated   as   a  virtue.      During   the 
diffusion  of    the   model   through    the    organization,    a   hierarcy   of   models  may   be 
required   since   perspectives   and   data   bases   change   at   different    levels    in 
the   organization.      The   information   system's  model  bank  will   contain  models 
differentiated  by   decision    and    level. 

Experience  with  managerial   implementation  has   often    indicated    that 
acceptance    is   most  easily  obtained    from   the   decision  makers   who   "suffered" 
through    the   development.      This   implies    that   diffusion   horizontally  and 
vertically   in    the    organization  will   be   most   successful  when  new  managers 
are   involved  and    feel   that    they   are   building   their   own  model.       It   may   be 

best   to  ef ficiently"re-invent   the   wheel"  with   a  model   building   system 

39 
structured   around   the   initial  model.  Not   only  will    this   promote   commit- 

ment,   understanding,    and   a   feeling   of    control    in    the  managers,    but    it   also 
provides    benefits    for   the   manager.      Some   managers    feel    that    the   exercise 
of   model   specification    and  building   are    as   valuable   as    computerized    usage. 
Figure    6  shows   a  hypothesized   plot   of   perceived  managerial  value   versus    the 
model  building   evolution.      High    initial    value   accrues    to    the   manager   as  h2 
structures    and   internalizes   a  previously   unstructured  problem.      He    is 
updating     his   mental  model    and  usually   gains    from   the   new  more    complete 

and  analytical    formulation.      Additional   benefits   of   better  understanding, 
forecasting,    and   decision  making   occur   in    computerized  model   estimation 
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and  use.   Diminishing  returns  to  increased  model  detail  then  set  in  due  to 
increasing  costs  and  decreasing  marginal  utility  to  the  manager.   Eventually, 
the  marginal  returns  could  become  negative  as  excessive  detail  causes  loss 
of  managerial  understanding  and  control  and  as  costs  of  estimating  and  running 
the  model  increase  rapidly. 

Another  factor  in  determining  marginal  benefits  is  the  time  required 
from  managers.   If  this  can  be  minimized, marginal  returns  and  acceptance 
will  be  higher.   During  the  process  of  re-inventing  and  customizing  the 
commitment  of  managerial  time  can  be  reduced  if  customizing  is  largely 
selecting  options  from  an  extensive  set  of  compatible  modules.  Then  the 
model  builder  can  build  on-line  "his"  model.   Time  efficiency  during  con- 
tinuing model  use  can  be  obtained  by  expressing  the  model  concisely  in  a 
non-computer  form.   For  example,  a  planning  log  documenting  model  runs  and 
tracking  efforts  is  useful  since  the  manager  can  use  technical  assistants  to 
execute  his  desired  runs  and  summarize  them  in  a  concise  form.   Although  on-line 
models  are  critical  to  managerial  feelings  of  control  and  understanding, 
most  managers  tire  of  physically  running  the  model  on  a  continuing  basis. 
However,  they  want  the  capability  of  quickly  making  runs  on  special  occasions. 

Maintaining  creativity  during  continuing  use  is  an  important  concept. 
Models  do  not  create,  they  (inly  execute  what  has  been  structured.   The  model 
and  computer  should  do  what  they  are  best  at  and  man  should  do  what  he  is  best 
at.   If  the  model  reduces  th3  burden  of  routine  work  the  manager  will  have 
more  time  to  create.   The  mooel  will  help  him  explore  the  implications  of 
such  innovation,  but  the  man  must  respond.   This  creative  push  can  come 
by  goals  which  demand  creative  innovation  to  achieve, by  a  committee  which 
tries  to  create  new  decision  alternatives  not  now  in  the  model,  or  by  an 
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40 

explicit  document  requiring  an  identification  of  creative  alternatives. 

During  continuing  use  of  most  systems,  a  long  run  sustaining  effort  is 
required  to  respond  to  changing  needs,  train  new  personnel,  and  maintain 
commitment.   Problem  environments  change  and  usually  managers  and  or  staff 
have  high  turnover  rates.   This  turnover  implies  that  customizing  and  training 
will  be  a  continuing  task.   The  model  building,  OR,  and  information  systems 
personnel  turnover  is  sometimes  higher  than  managerial  turnover.   This 
combined  with  a  general  shortage  of  people  capable  of  supporting  model 
implementation  implies  that  explicit  programs  are  required  to  recruit  and 
train  the  model  building  and  implementing  groups. 

The  comments  in  this  section  argue  that  it  would  be  difficult  to  view 

implementation  as  complete,  but  rather  that  continuing  inputs  will  be 

required  to  assure  that  the  models  meet  managers'  needs  and  are  used  by 

them.   If  a  model  is  productive,  over  time  the  procedures  will  become 

institutionalized.   For  example,  requiring  model  support  for  plans  and 

instituting  a  protocol  of  quarterly  tracking  reports  will  facilitate 

implementation.   The  dynamics  of  the  decision  environment  and  organization 

will  dictate  an  adaptive  approach  to  implementation  rather  than  a  final 

institutionalization. 

It  is  clear  that  if  the  process  proposed  in  this  paper  is  followed, 

continuing  use  of  models  will  change  decision  procedures  and  organizational 

relationships.   Explicit  efforts  should  be  made  to  monitor  these  effects 

and  take  action  to  assure  effective  planned  change  in  the  organization  and 

effective  implementation  of  models  to  produce  better  decision  making. 

Concluding  Comments 
This  paper  has  advocated  a  process  of  building  models  that  are  to 
be  used  by  managers  to  improve  their  decision  making.   It  is  inductively 
based  on  experience  and  deductively  on  the  published  literature.   The 
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experience  was  gained  by  failures  and  successes  in  building  and  imple- 

41 
menting  new  product  models.    The  process  became  explicit  during  a  three 

42 
year  implementation  of  an  advertising  model.    The  process  was  followed 

step  by  step  in  a  family  planning  strategy  model  which  is  achieving 

43 
successful  implementation.     This  by  no  means  establishes  the  validity 

44 
of  the  process,  but  it  does  provide  confirmatory  evidence.   Little, 

45  45         47 

Adleman,    Scott  Morton,  and  Gorry   have  also  found  success  with  approaches  utilizing 

similar  process  concepts.   Research  is  needed  to  experimentally  find  the 
best  process.   This  paper  proposes  one  experimental  treatment.   Other 
alternative  procedures  must  be  stated  and  suitable  comparative  tests 
devised. 

Within  the  proposed  process  many  research  questions  are  unanswered. 
First,  modeling  remains  an  art  and  research  into  the  process  of  structuring 
complex  situations  and  selecting  from  diverse  model  alternatives  is  needed. 
The  estimation  of  parameters  requires  non-linear  procedures  in  many  situations 
and  statistics  are  needed  for  these  cases.   Work,  is  needed  to  allow  better 
judgmental  inputs  and  improved  procedures  to  combine  empirical  and  sub- 
jective inputs  in  the  complex  models  used  in  management  decision  making. 
Research  on  tracking  is  needed  to  separate  the  causes  of  deviation  between 
actual  and  predicted  results.   Generalization  of  Tliiel's  U  or  the  Janis 
coefficient  is  an  attractive  first  step.   Research  is  needed  to  determine 
better  methods  to  facilitate  entry,  problem  finding, and  continuing  use. 
This  will  require  inter-disciplinary  work  between  behavioral  and  management 
scientists  to  find  the  best  approach  to  the  planned  organizational  change 
required  by  models.   Research  is  needed  to  develop  procedures  to  explicitly 
measure  the  personal,  organizational!,  and  decision  impact  of  models. 
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For  example,  is  the  hypothesized  relationship  between  model  building 
detail  and  perceived  benefits  in  figure  6  true?   A  final  behavioral 
project  is  to  research  the  diffusion   of  innovation  process  in  terras  of 
characteristics  of  organizations  and  decision  makers  who  are  most  likely 
to  accept  models. 

Another  issue  raised  by  the  methodology  is  ethics.   It  is  possible 
with  the  degrees  of  freedom  in  fitting  and  the  complexity  of  the  math- 
ematical translation  of  the  model   that  the  model  builder  could  exert 
undue  influence.   Indeed,  with  an  understandable  and  controllable  model, 
the  manager  and/or  model  builder  can  produce  whatever  answers  they  want. 
The  explicitness  of  the  model  assumption  and  the  tracking  procedures 
provide  some  protection  against  such  manipulation,  but  professional  ethicS 
are  required.   This  can  be  approached  by  appropriate  academic  training 
of  model  builders  that  includes  "hands  on"  training  in  how  to  build  models 
and  develop  professional  standards  for  "fitting"  and  managerial  use.   This 

is  a  particularily  difficult  issue  in  view  of  the  political  roles  the  model 

49 
builder  and  manager  may  be  called  upon  to  play. 

It  is  hoped  that  this  emerging  process  of  building  models  backed  by 

professional  model  builders  and  additional  research  work  will  develop  into 

a  methodology  which  will  make  real  to  managers  the  promise  of  management 

science, models,  and  information  systems. 
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